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Abstract 


A  current  Applied  Research  Project  (ARP)  of  the  Collaborative  Performance  and  Learning  Section 
(CPL)  at  Defence  Research  and  Development  Canada  (DRDC)  Toronto  explores  moral  and  ethical 
decision-making  (MEDM)  in  Canadian  Forces  (CF)  operations  (16kk,  16re02).  A  major  focus  in 
this  program  of  research  has  been  field  studies  conducted  at  the  Peace  Support  Training  Centre 
(PSTC)  Canadian  Forces  Base  (CFB)  in  Kingston.  CF  personnel  preparing  to  deploy  as  United 
Nations  Military  Observers  (UNMOs  or  Mil  Ohs)  undertake  a  number  of  training  activities  meant 
to  enhance  and  develop  their  skills  particular  to  Peace  Support  Operations  (PSO).  Over  the  course 
of  a  month  long  training  program,  CF  personnel  receive  intensive  preparation,  using  a  combination 
of  classroom  training  and  scenario-based  training.  One  such  scenario-based  exercise  simulates  a 
human  rights  violation.  In  this  scenario,  unarmed  trainees  attempt  to  negotiate  the  lives  of  civilians 
who  are  being  verbally  and  physically  abused  by  two  armed  police.  Our  research  program  to  date 
has  examined  a  number  of  psychological  processes  during  this  scenario,  such  as  aspects  of  moral 
intensity  (physical  proximity  of  the  victim,  MFFS  I),  emotion  (MFFS  II)  and  teamwork  (MFFS 
III).  The  current  work  provides  PSTC  instructor  requested  materials  that  are  drawn  from  the  results 
of  these  studies. 

First,  we  created  a  set  of  PowerPoint  slides  that  detail  the  most  critical  findings  in  the  three  field 
studies.  Second,  we  generated  a  list  of  the  open-  and  closed-ended  questions  that  trainees  asked 
during  the  scenarios  across  all  three  studies  (MFFS  I,  MFFS  II,  MFFS  III).  This  list  is  provided  as 
a  means  to  underscore  PSTC  instruction  to  trainees,  i.e.,  emphasizing  the  use  of  open-ended 
questions  for  eliciting  situational  awareness.  Tables  have  been  created  that  include  specific 
examples  of  open-  and  closed-  ended  questions  seeking  both  general  and  specific  information.  A 
separate  table  was  constructed  in  which  the  common  themes  were  extracted  from  these  questions, 
as  well  as  prototypic  examples  of  questions  asked  by  the  trainees.  The  third,  and  final  section  of 
this  report,  includes  the  preparatory  activities  that  lay  the  ground  work  for  a  PSTC  training 
evaluation.  Specifically,  the  evaluation  is  intended  to  explore  how  well  the  training  that  UNMOs 
receive  at  the  PSTC  during  the  Mil  Ohs  course  translates  into  actual  operational  experience.  This 
report  concludes  with  possible  next  steps  toward  the  evaluation  and  a  plan  for  data  collection. 


Resume 


Un  Programme  de  recherches  appliquees  (PRA)  en  cours  de  la  Section  de  la  recherche  concertee 
sur  le  rendement  et  I'apprentissage  (Section  RCRA)  a  Recherche  et  developpement  pour  la  defense 
Canada  (RDDC)  Toronto  explore  la  prise  de  decisions  morales  et  ethiques  (PD ME)  dans  le  cadre 
des  operations  des  Forces  canadiennes  (FC)  (16kk,  16re02).  Fes  etudes  sur  le  terrain  menees  au 
Centre  de  formation  pour  le  soutien  de  la  paix  (CFSP)  a  la  base  des  Forces  canadiennes  (BFC)  de 
Kingston  sont  Tun  des  importants  champs  d’interet  de  ce  programme  de  recherches.  Fes  membres 
des  FC  qui  se  preparent  a  partir  en  mission  en  tant  qu’observateurs  militaires  des  Nations  Unies 
(UNMO  ou  Ob  mil)  entreprennent  un  certain  nombre  d’activites  d’entrainement  destinees  a 
ameliorer  et  developper  les  competences  propres  aux  Operations  de  paix  des  Nations  Unies 
(OPNU).  Pendant  un  mois,  les  membres  des  FC  suivent  un  programme  de  preparation  intensif 
faisant  appel  a  une  combinaison  de  formation  en  classe  et  d’ instruction  en  situation.  F’un  des 
exercices  proposes  met  en  scene  une  situation  de  violation  des  droits  de  la  personne.  Dans  ce 
scenario,  les  stagiaires  non  armes  tentent  de  negocier  les  vies  de  civils  qui  sont  victimes  de 
violences  physiques  et  verbales  de  la  part  de  deux  policiers  armes.  Notre  programme  de  recherche 
a  examine  jusqu’ici  un  certain  nombre  de  processus  psychologiques  au  cours  de  ce  scenario,  tels 
que  les  aspects  de  Tintensite  morale  (proximite  physique  de  la  victime,  MEFS  I),  emotion 
(MEFS  II)  et  travail  d’equipe  (MEFS  III).  Fes  travaux  foumis  actuellement  par  les  instructeurs  du 
CFSP  demandent  des  documents  qui  sont  tires  des  resultats  de  ces  etudes. 

Premierement,  nous  avons  cree  une  serie  de  diapositives  PowerPoint  qui  montrent  en  detail  les 
decouvertes  les  plus  importantes  des  trois  etudes  sur  le  terrain.  Deuxiemement,  nous  avons  redige 
la  liste  des  questions  ouvertes  (a  reponse  libre)  et  fermees  (a  reponse  preetablie)  que  les  stagiaires 
ont  posees  durant  le  scenario  lors  des  trois  etudes  (MEFS  I,  MEFS  I  et  MEFS  III).  Cette  liste  est 
fournie  afm  de  mettre  en  evidence  T instruction  du  CFSP  aux  stagiaires,  c’est-a-dire  mettre  en 
valeur  Putilisation  de  questions  ouvertes  pour  provoquer  une  connaissance  de  la  situation.  Des 
tableaux  contenant  des  exemples  precis  de  questions  ouvertes  et  fermees  destinees  a  obtenir  des 
renseignements  generaux  et  particuliers  ont  etc  elabores.  On  a  egalement  prepare  un  tableau 
distinct  dans  lequel  les  themes  communs  ont  etc  extraits  de  ces  questions;  ce  tableau  contient  aussi 
des  exemples  prototypiques  des  questions  posees  par  les  stagiaires.  Fa  troisieme  et  demiere  section 
du  present  rapport  comprend  les  activites  preparatoires  qui  jettent  les  bases  d’une  evaluation  de 
I’instruction  du  CFSP.  Plus  precisement,  cette  evaluation  a  pour  but  de  decouvrir  dans  quelle 
mesure  I’instruction  rcfue  par  les  UNMO  au  CFSP  dans  le  cadre  du  cours  d’observateur  militaire 
se  traduit  en  une  reelle  experience  operationnelle.  Fe  present  rapport  se  termine  en  presentant  les 
prochaines  etapes  menant  a  cette  evaluation  ainsi  qu’un  plan  pour  la  collecte  de  donnees. 
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Executive  Summary 


A  current  Applied  Research  Project  (ARP)  of  the  Collaborative  Performance  and  Learning  Section 
(CPL)  at  Defence  Research  and  Development  Canada  (DRDC)  Toronto  explores  moral  and  ethical 
decision-making  (MEDM)  in  Canadian  Forces  (CF)  operations  (16kk,  16re02).  A  major  focus  in 
this  program  of  research  has  been  field  studies  conducted  at  the  Peace  Support  Training  Centre 
(PSTC)  Canadian  Forces  Base  (CFB)  in  Kingston.  CF  personnel  preparing  to  deploy  as  United 
Nations  Military  Observers  (UNMOs  or  Mil  Ohs)  undertake  a  number  of  training  activities  meant 
to  enhance  and  develop  their  skills  particular  to  Peace  Support  Operations  (PSO).  Over  the  course 
of  a  month  long  training  program,  CF  personnel  receive  intensive  preparation  in  basic  survival 
(e.g.,  mine  awareness,  first  aid)  as  well  as  in  more  complex  skills  (e.g.,  negotiation  and  risk 
assessment),  using  a  combination  of  classroom  training  and  scenario-based  training.  One  such 
scenario-based  exercise  simulates  a  human  rights  violation.  In  this  scenario,  unarmed  trainees 
encounter  armed  police  (a  sergeant  and  constable)  verbally  and  physically  abusing  two  civilians 
and  forcing  them  to  dig  what  could  be  their  own  graves.  Trainees  are  challenged  to  use  the 
negotiation  skills  they  have  learned  in  classroom  lectures  and  exercises  to  fulfil  their  mission 
mandate,  and  to  advocate  for  the  fair  treatment  of  the  civilians  in  an  effort  to  promote  a  successful 
resolution  for  all  parties. 

Our  research  program  to  date  has  examined  a  number  of  psychological  processes  during  the  human 
rights  violation  scenario.  In  the  first  field  study  (MFFS  I),  research  examined  the  impact  of  a 
dimension  of  moral  intensity  on  MFDM  and  negotiation  behaviour  by  varying  the  physical 
proximity  of  one  of  the  victims  (female)  in  the  scenario  to  the  trainees.  The  second  field  study 
(MFFS  II)  conducted  during  the  human  rights  violation  scenario  examined  the  impact  of  emotion 
on  the  negotiation  behaviour  of  the  trainees.  The  third  field  study  (MFFS  III)  investigated  the 
impact  of  teamwork  on  negotiation  behaviour  and  the  perceived  level  of  participation  of  all  team 
members  in  the  scenario.  The  current  work  provides  PSTC  instructor  requested  materials  that  are 
drawn  from  the  results  of  these  studies. 

First,  we  created  a  set  of  PowerPoint  slides  that  detail  the  most  critical  findings  in  the  three  field 
studies.  Second,  we  generated  a  list  of  the  open-  and  closed-ended  questions  that  trainees  asked 
during  the  scenarios  across  all  three  studies  (MFFS  I,  MFFS  II,  MFFS  III).  This  list  is  provided  as 
a  means  to  underscore  PSTC  instruction  to  trainees,  i.e.,  emphasizing  the  use  of  open-ended 
questions  for  eliciting  situational  awareness.  Tables  have  been  created  that  include  specific 
examples  of  open-  and  closed-  ended  questions  seeking  both  general  and  specific  information.  A 
separate  table  was  constructed  in  which  the  common  themes  were  extracted  from  these  questions, 
as  well  as  prototypic  examples  of  questions  asked  by  the  trainees.  PSTC  leaders  and  instructors 
have  expressed  considerable  interest  in  evaluating  their  training  course  to  determine  how  relevant 
the  training  is  to  current  UNMO  operations.  The  third,  and  final  section  of  this  report,  includes  the 
preparatory  activities  that  lay  the  ground  work  for  the  PSTC  training  evaluation.  Specifically,  the 
evaluation  is  intended  to  explore  how  well  the  training  that  UNMOs  receive  at  the  PSTC  during  the 
Mil  Ohs  course  translates  into  actual  operational  experience.  To  start  the  process  of  designing  this 
evaluation,  the  research  team  convened  a  two-hour  discussion  session  with  3  PSTC  course 
instructors  at  CFB  Kingston  to  determine  the  most  critical  objectives  of  the  Mil  Ohs  course.  An 
evaluation  framework  was  developed  as  a  result  of  the  discussion.  Next  steps  for  the  PSTC  training 
evaluation  are  proposed. 


Sommaire 


Un  Programme  de  recherches  appliquees  (PRA)  en  cours  de  la  Section  de  la  recherche  concertee 
sur  le  rendement  et  I'apprentissage  (Section  RCRA)  a  Recherche  et  developpement  pour  la  defense 
Canada  (RDDC)  Toronto  explore  la  prise  de  decisions  morales  et  ethiques  (PD ME)  dans  le  cadre 
des  operations  des  Forces  canadiennes  (FC)  (16kk,  16re02).  Fes  etudes  sur  le  terrain  menees  au 
Centre  de  formation  pour  le  soutien  de  la  paix  (CFSP)  a  la  base  des  Forces  canadiennes  (BFC)  de 
Kingston  sont  Tun  des  importants  champs  d’interet  de  ce  programme  de  recherches.  Fes  membres 
des  FC  qui  se  preparent  a  partir  en  mission  en  tant  qu’observateurs  militaires  des  Nations  Unies 
(UNMO  ou  Ob  mil)  entreprennent  un  certain  nombre  d’activites  d’entrainement  destinees  a 
ameliorer  et  developper  les  competences  propres  aux  Operations  de  paix  des  Nations  Unies 
(OPNU).  Pendant  un  mois,  les  membres  des  FC  suivent  un  programme  de  preparation  intensif  en 
survie  elementaire  (p.  ex.,  instruction  sur  les  mines,  premiers  soins)  ainsi  que  de  la  formation  axee 
sur  le  perfectionnement  de  competences  plus  complexes  (p.  ex.,  negociation  et  evaluation  des 
risques).  Ce  programme  fait  appel  a  une  combinaison  de  formation  en  classe  et  d’ instruction  en 
situation.  F’un  des  exercices  proposes  met  en  scene  une  situation  de  violation  des  droits  de  la 
personne.  Dans  ce  scenario,  les  stagiaires  non  armes  rencontrent  des  policiers  armes  (un  sergent  et 
un  constable)  qui  agressent  verbalement  et  physiquement  deux  civils  tout  en  les  forfant  a  creuser  ce 
qui  pourrait  etre  leur  propre  tombe.  Fe  defi  des  stagiaires  consiste  a  utiliser  les  techniques  de 
negociation  qu’ils  ont  apprises  en  classe  et  lors  d’ exercices  afin  de  remplir  le  mandat  de  leur 
mission  et  de  preconiser  le  traitement  equitable  des  civils,  le  tout  dans  un  effort  visant  a  encourager 
un  reglement  de  la  situation  qui  convienne  a  toutes  les  parties. 

Jusqu’ici  notre  programme  de  recherche  a  examine  un  certain  nombre  de  processus  psychologiques 
presents  lors  du  scenario  portant  sur  la  violation  des  droits  de  la  personne.  Dans  la  premiere  etude 
sur  le  terrain,  (MEFS  I),  les  chercheurs  ont  examine  les  repercussions  d’une  dimension  de 
I’intensite  morale  sur  la  prise  de  decisions  morales  et  ethiques  (PD ME)  et  sur  le  comportement  des 
stagiaires  en  matiere  de  negociation  en  faisant  varier  la  distance  physique  entre  Tune  des  victimes 
(la  femme)  et  les  stagiaires.  Fa  deuxieme  etude  sur  le  terrain  (MEFS  II),  menee  dans  le  cadre  du 
scenario  portant  sur  la  violation  des  droits  de  la  personne,  s’est  penchee  sur  les  repercussions  des 
emotions  sur  le  comportement  des  stagiaires  en  matiere  de  negociation.  Fa  troisieme  etude  sur  le 
terrain  (MEFS  III)  s’interessait  aux  repercussions  du  travail  d’equipe  sur  le  comportement  des 
stagiaires  lors  de  la  negociation  ainsi  que  sur  le  niveau  de  participation  perfu  de  tons  les  membres 
de  I’equipe  dans  le  scenario.  Fe  present  travail  foumit  la  documentation  demandee  par  les 
instructeurs  du  CFSP,  documentation  qui  est  tiree  des  resultats  de  ces  etudes. 

Premierement,  nous  avons  cree  une  serie  de  diapositives  PowerPoint  qui  montrent  en  detail  les 
decouvertes  les  plus  importantes  des  trois  etudes  sur  le  terrain.  Deuxiemement,  nous  avons  redige 
la  liste  des  questions  ouvertes  (a  reponse  libre)  et  fermees  (a  reponse  preetablie)  que  les  stagiaires 
ont  posees  durant  le  scenario  lors  des  trois  etudes  (MEFS  I,  MEFS  I  et  MEFS  III).  Cette  liste  est 
foumie  afm  de  mettre  en  evidence  I’instruction  du  CFSP  aux  stagiaires,  c’est-a-dire  mettre  en 
valeur  I’utilisation  de  questions  ouvertes  pour  provoquer  une  connaissance  de  la  situation.  Des 
tableaux  contenant  des  exemples  precis  de  questions  ouvertes  et  fermees  destinees  a  obtenir  des 
renseignements  generaux  et  particuliers  ont  etc  elabores.  On  a  egalement  prepare  un  tableau 
distinct  dans  lequel  les  themes  communs  ont  etc  extraits  de  ces  questions;  ce  tableau  contient  aussi 
des  exemples  prototypiques  des  questions  posees  par  les  stagiaires.  Fes  dirigeants  et  les 
instructeurs  du  CFSP  ont  manifeste  beaucoup  d’interet  envers  revaluation  de  la  formation  qu’ils 
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donnent.  Cette  evaluation  permettra  de  determiner  le  niveau  de  pertinence  de  cette  instruction  par 
rapport  aux  operations  menees  actuellement  par  les  UNMO.  La  troisieme  et  demiere  section  du 
present  rapport  comprend  les  activites  preparatoires  qui  jettent  les  bases  d’une  evaluation  de 
I’instruction  du  CFSP.  Plus  precisement,  cette  evaluation  a  pour  but  de  decouvrir  dans  quelle 
mesure  I’instruction  rcfue  par  les  UNMO  au  CFSP  dans  le  cadre  du  cours  d’observateur  militaire 
se  traduit  en  une  reelle  experience  operationnelle.  Pour  demarrer  le  processus  de  conception  de 
revaluation,  les  membres  de  I’equipe  de  recherche  ont  tenu  une  seance  de  discussion  de  deux 
heures  avec  trois  instructeurs  du  CFSP  a  la  BFC  Kingston  afin  de  determiner  les  objectifs  les  plus 
importants  du  cours  d’observateur  militaire.  A  Tissue  de  ces  discussions,  un  cadre  d’evaluation  a 
etc  mis  au  point.  Des  suggestions  sont  faites  concemant  les  prochaines  etapes  du  processus 
d’evaluation  de  I’instruction  donnee  au  CFSP. 


1.  Introduction 


An  ongoing  research  initiative  of  the  Collaborative  Performance  and  Learning  Section  (CPL)  at 
Defence  Research  and  Development  Canada  (DRDC)  Toronto  has  explored  moral  and  ethical 
decision-making  (MEDM)  in  Canadian  Forces  (CF)  operations.  A  major  focus  in  this  program  of 
research  has  been  field  studies  conducted  at  the  Peace  Support  Training  Centre  (PSTC)  Canadian 
Forces  Base  (CFB)  Kingston.  CF  personnel  preparing  to  deploy  as  United  Nations  Military 
Observers  (UNMO  or  Mil  Ohs)  undertake  a  number  of  training  activities  meant  to  enhance  and 
develop  their  skills  particular  to  Peace  Support  Operations  (PSO).  Over  the  course  of  a  month  long 
training  program,  CF  personnel  receive  critical  training  in  basic  survival  (e.g.,  mine  awareness, 
first  aid)  as  well  as  in  more  complex  skills  (e.g.,  negotiation  and  risk  assessment),  using  a 
combination  of  classroom  training  and  scenario-based  training.  One  such  scenario-based  exercise 
simulates  a  human  rights  violation.  In  this  scenario,  unarmed  trainees  encounter  armed  police  (a 
sergeant  and  constable)  verbally  and  physically  abusing  two  civilians  and  forcing  them  to  dig  what 
could  be  their  own  graves.  The  civilians  plead  for  their  lives,  continuously  declaring  their 
innocence  and  imminent  death  should  the  trainees  leave.  The  trainees  are  kept  at  a  distance 
(approximately  60  feet)  from  the  two  civilians  by  the  lead  police  officer,  the  sergeant  (Sgt).  Trainee 
teams,  led  by  a  designated  team  leader  and  consisting  of  2  to  4  members,  must  negotiate  with  him 
(the  Sgt)  only.  Team  members  observe  the  proceedings  between  the  Sgt  and  the  team  leader, 
offering  their  assistance  and  input  from  time  to  time.  Trainees  are  challenged  to  use  the  negotiation 
skills  they  have  learned  in  classroom  lectures  and  exercises  to  fulfil  their  mission  mandate,  and  to 
advocate  for  the  fair  treatment  of  the  civilians  in  an  effort  to  promote  a  successful  resolution  for  all 
parties. 

Our  research  program  to  date  has  examined  a  number  of  psychological  processes  during  the  human 
rights  violation  scenario.  In  the  first  field  study  (MFFS  I),  research  examined  the  impact  of  moral 
intensity  (i.e.,  the  salience  and  vividness  of  the  characteristics,  such  as  proximity,  of  a  moral  issue; 
Jones,  1991)  on  MFDM  and  negotiation  behaviour  by  varying  the  physical  proximity  of  one  of  the 
victims  (female)  in  the  scenario  to  the  trainees.  The  second  field  study  (MFFS  II)  conducted  during 
the  human  rights  violation  scenario  examined  the  impact  of  emotion  on  the  negotiation  behaviour 
of  the  trainees.  Specifically,  trainees  either  confronted  a  neutral  negotiation  partner  or  an  angry 
negotiation  partner  to  determine  if  trainees  confronted  with  an  angry  Sgt  would  either  “take  on”  the 
Sgf  s  emotion  (known  as  social  contagion;  van  Kleef,  De  Dreu,  and  Manstead,  2004)  or  act  more 
strategically  to  try  to  diffuse  the  situation  by  lessening  one’s  demands  and  increasing  offers  and 
concessions  (known  as  the  strategic  choice  hypothesis;  van  Kleef  et  al.,  2004).  The  third  field  study 
(MEFS  III)  investigated  the  impact  of  teamwork  on  negotiation  behaviour  and  the  perceived  level 
of  participation  of  all  team  members  in  the  scenario.  Specifically,  research  examined  particular 
team  behaviours  emerging  in  instances  in  which  team  members  briefly  met  together  to  discuss  their 
options  in  the  scenario  free  of  interference  from  the  Sgt.  The  intent  of  these  three  studies  together 
was  to  vary  some  aspect  of  all  three  parties’  behaviour  and  determine  the  resulting  effects  on  the 
negotiation.  In  MEFS  I,  the  victims  behaviour  was  manipulated.  In  MEFS  II,  the  police’s 
behaviour  was  manipulated.  And  in  MEFS  III,  the  trainees’  behaviour  was  manipulated.  Data 
analysis  investigated  the  impact  of  these  manipulations  on  UNMO  trainees  MEDM  and  negotiation 
behaviour. 

Following  the  completion  of  each  field  study,  the  research  team  shared  the  results  with  PSTC 
leaders  and  instructors.  During  the  MEFS  III  briefing,  the  PSTC  Commandant  and  senior  instructor 
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asked  for  the  main  findings  of  each  of  the  studies  to  be  briefly  summarized  in  a  PowerPoint 
presentation  to  assist  them  in  integrating  this  information  into  the  course  curriculum.  As  well,  they 
thought  that  it  would  be  useful  to  have  a  list  of  the  open-  and  closed-ended  questions  that  trainees 
asked  during  the  scenarios  as  a  vivid  means  to  underscore  PSTC  instructions  to  trainees 
emphasizing  the  use  of  open-ended  questions  as  the  most  effective  means  of  eliciting  situational 
awareness.  Finally,  PSTC  leaders  and  instructors  have  expressed  considerable  interest  in  evaluating 
their  training  course  to  determine  how  relevant  the  training  is  to  current  UNMO  operations. 
Specifically,  how  relevant  is  the  training  to  operations  and  how  could  it  be  made  more  relevant? 
The  following  work  then  includes  the  main  findings  from  the  three  field  studies  and  a  table  of  the 
open  and  closed-ended  questions.  It  also  includes  the  initial  activities  for  a  more  holistic  evaluation 
of  PSTC  training  in  future  work. 


2.  Method 


1 .  PowerPoint  Presentation 

The  first  component  of  this  project  included  creating  a  set  of  PowerPoint  slides  that  detail  the  most 
critical  findings  in  the  three  field  studies.  For  each  study,  the  slides  include  a  brief  description  of 
the  field  study,  the  experimental  manipulation,  and  the  main  findings.  The  PowerPoint  presentation 
concludes  with  “Lessons  Learned”  from  the  three  studies.  Because  the  slides  are  meant  to  be 
integrated  into  the  PSTC  training  curriculum,  a  research  team  member  ensured  that  the  information 
included  on  the  slides  were  succinct  and  relevant  for  PSTC  training  purposes.  Flowever,  it  was 
important  to  provide  as  much  context  in  the  slides  so  that  future  PSTC  leaders  and  instructors  can 
understand  enough  of  the  research  purpose  and  results. 

2.  Open-  and  Closed-ended  Questions 

At  the  end  of  each  Field  Study,  the  Contractor  and  the  DRDC  scientific  authority  (SA)  briefed  the 
Commandant  on  the  findings.  Across  these  briefings,  two  particular  codes  that  interested  PSTC 
instructors  were  1)  asking  situation  specific  information  about  what  is  happening  in  the  scenario 
and  2)  asking  general  information  not  directly  relevant  to  the  specific  situation.  The  former 
referred  to  questions  in  the  immediate  vicinity  (e.g.,  “What  are  those  two  people  doing?”),  while 
the  latter  referred  to  questions  outside  the  physical  area  of  the  scenario  (e.g.,  “Flow  long  have  you 
lived  in  these  parts?”).  For  MEFS  I  and  II,  we  had  only  coded  questions  based  on  the  criteria 
situation  specific  and  situation  general.  Flowever,  PSTC  instructors  showed  an  interest  in 
determining  if  trainees  were  asking  open-  or  closed-ended  questions,  as  trainees  are  instructed  to 
exercise  the  former  over  the  latter  when  negotiating.  Open-ended  questions  are  encouraged  in 
training  because  they  yield  more  information  than  closed-ended  questions,  which  often  result  in 
“yes”  or  “no”  answers.  While  briefing  the  instructors  on  MEFS  II,  they  asked  if  we  could  code  also 
for  open-ended  versus  closed-ended  questions  for  the  upcoming  MEFS  III  trial.  Briefing  and 
reporting  on  MEFS  III,  therefore,  included  tables  of  open-  and  closed-ended  questions.  PSTC 
instructors  thought  having  the  list  of  questions  categorized  in  this  way  was  useful  for  training 
purposes.  After  reviewing  the  tables,  PSTC  instructors  asked  if  we  could  go  back  to  the  data  from 
MEFS  I  and  MEFS  II  and  produce  similar  tables  (i.e.,  situation  specific  open-  and  closed-ended 
questions  and  general  open-  and  closed-ended  questions).  For  this  contract,  a  research  member 
watched  all  the  videos  from  the  previous  studies  and  documented  the  open-  and  closed-ended 
questions  from  MEFS  I  and  MEFS  IT 

The  original  datasets  used  for  MEFS  I  and  II  included  an  excel  spreadsheet  listing  all  of  the 
behavioural  codes,  including  the  two  codes  of  interest  (i.e.,  seeking  situation  specific  information 
and  seeking  general  information).  A  member  of  the  research  team  sorted  the  original  file  by  code 
and  then  created  a  new  excel  spreadsheet  listing  only  the  two  behaviours  of  interest.  Also  included 
in  the  new  spreadsheet  was  the  study  date  (month  and  year),  the  session  number,  the  team  and  team 
leader  identifier,  the  condition  (experimental  or  baseline),  open-  versus  closed-ended  question 
code,  the  start  and  end  time  the  question  appeared  in  the  particular  scenario,  who  in  the  scenario 
asked  the  question  (e.g.,  team  leader  or  a  team  member)  and  to  whom  it  was  directed  (e.g.,  Sgt, 
victims,  UNMO,  etc.),  and  a  word-for-word  account  of  the  questioning.  These  verbatim  accounts 
as  well  as  open-  versus  closed-ended  questions  were  separated  by  study  and  incorporated  into  table 
format,  which  included  those  already  broken  out  for  MEFS  III.  Any  duplicated  questions,  questions 
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that  did  not  make  sense,  or  questions  that  were  incomplete  because  the  speaker  was  interrupted 
were  not  included  in  the  table. 

In  total,  five  tables  were  created  for  this  requirement.  These  include: 

•  Closed-ended  questions  seeking  general  information; 

•  Closed-ended  questions  seeking  situation  specific  information; 

•  Open-ended  questions  seeking  general  information; 

•  Open-ended  questions  seeking  situation  specific  information;  and 

•  Common  questioning  themes  and  prototypic  examples. 

Within  these  tables,  common  themes  emerged.  These  were  further  broken  down  into  sub-themes.  A 
fifth  table  was  created  that  included  examples  extracted  from  the  original  four  tables.  The  final 
table  includes  the  most  frequently  occurring  themes  and  prototypic  examples  of  open-  and  closed- 
ended  questions  from  MEFS  I,  II  and  III. 

3.  Preparatory  Activities  for  PSTC  Training  Evaluation 

The  third,  and  final  section  of  this  report,  includes  the  preparatory  activities  that  lay  the  ground 
work  for  the  PSTC  training  evaluation.  Specifically,  this  evaluation  is  intended  to  explore  how 
relevant  the  current  training  activities  during  the  PSTC  Mil  Ohs  course  map  on  to  the  requirements 
in  current  UNMO  operations.  To  start  the  process  of  designing  this  evaluation,  the  research  team 
convened  a  two-hour  discussion  session  with  3  PSTC  course  instructors  at  CEB  Kingston. 
Instructors  were  identified  on  the  basis  of  their  rank  (one  major  and  two  captains)  and  experience 
instructing  the  Mil  Ohs  course.  Before  this  meeting,  the  research  team  developed  a  number  of 
pertinent  questions  in  order  to  facilitate  the  discussion.  Topics  included  the  kind  of  information 
PSTC  staff  hoped  to  gain  from  the  evaluation  and  how  they  intended  to  use  it;  determining  the 
long-term  outcomes  for  trainees  associated  with  the  Mil  Ohs  course;  and  identifying  the  critical 
training  activities  meant  to  further  the  long-term  outcomes  of  the  Mil  Ohs  course.  During  the 
discussion,  members  of  the  research  team  asked  questions  to  generate  discussion  and  to  clarify 
points  that  had  been  made.  The  discussion  with  PSTC  course  instructors  was  recorded  using  a  Sony 
Mini-disc  recorder.  The  information  gained  from  the  PSTC  Instructors  discussion  was  used  to 
develop  a  program  evaluation  framework.  This  framework  can  be  used  to  develop  measurements 
for  a  PSTC  training  evaluation  seeking  to  understand  the  relevance  of  the  course  material  to  actual 
United  Nations  deployments. 


3 


Deliverables 


3.1  Main  Research  Findings  from  Three  Field  Studies  (MEFS  I,  II,  III) 

The  following  section  includes  a  copied  set  of  PowerPoint  slides  that  detail  the  most  critical 
findings  in  the  three  field  studies  (the  contract  includes  the  PowerPoint  presentation).  Please  see 
Annex  A. 

3.2  Open-  and  Closed-ended  Questions  from  Three  Field  Studies 

The  following  section  includes  five  tables  showing  specific  categories  of  questions  that  Military 
Observers  asked  of  the  Sgt  during  the  human  rights  violation  training  scenario  at  PSTC.  These 
include: 


•  Closed-ended  questions  seeking  general  information; 

•  Closed-ended  questions  seeking  situation  specific  information; 

•  Open-ended  questions  seeking  general  information; 

•  Open-ended  questions  seeking  situation  specific  information;  and 

•  Common  questioning  themes  and  prototypic  examples. 

As  some  of  the  questions  were  asked  in  a  very  specific  context,  their  meaning  might  be  unclear. 
For  these,  further  context  is  added  in  brackets  as  necessary.  Table  1  shows  the  closed-ended 
questions  seeking  general  information  asked  by  trainees  throughout  the  three  field  studies. 
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Table  1:  Closed-ended  questions  seeking  general  information 


MEFSI 

MEFS  II 

MEFS  III 

Have  you  seen  any  [human  rights 
violations  in  the  area]? 

And  that's  [at  the  police  station  in 
town]  where  your  supervisor  is? 

Do  you  have  a  higher  boss,  someone 
to  talk  to  or  are  you  under  direct 
orders? 

[Is  the  police  station]  ]ust  down  this 
road? 

Do  you  have  children  at  all?  Family  in 
the  area? 

Is  [the  area]  getting  any  better  or 
worse  with  the  UN  around? 

How  long  have  you  been  working  for 
the  police? 

So  when’s  somebody  in  custody,  you 
can't  talk  to  him? 

How  many  guys  are  there,  it  must  be 
a  pretty  big  [prisoner]  camp? 

And  the  number  there  [for  the  police 
station],  the  local  phone  number? 

What  police  department  are  you 
from? 

There  must  be  a  lot  of  guys  out 
looking  for  the  PLA  guys? 

Do  you  have  contact  with  [your 
commander]? 

What  language  do  you  speak  other 
than  English? 

Have  you  seen  any  other  groups 
(terrorists)  around? 

[Pointing  to  the  Sgt’s  earplug] 

Do  you  have  some  music  there? 

Is  it  safe  for  us  to  go  straight  into  the 
village? 

How  long  have  you  been  with  the 

WIA? 

And  what's  the  name  of  the  town? 

Is  there  any  other  police  in  the  area 
than  yourselves? 

Do  you  know  a  phone  number  we  can 
reach  him  [Capt]  at? 

Are  you  from  the  national  police? 

What  kind  of  vehicles  are  you  going  to 
transport  [the  prisoners]  in? 

Did  it  happen  before  that  you  had  to 
deal  with  terrorists? 

How  many  kms  [do]  you  cover? 

Does  [the  inspector]  plan  to  come 
around  soon? 

Who’s  your  contact  at  the  police 
station? 

How  big  is  your  police  station? 

What's  the  name  of  the  town?  [where 
the  police  department  is  located] 

How  far  down  the  road  is  your  HQ? 

So  what  prison  do  you  work  for? 

Have  you  had  to  investigate  any 
[abuse  in  the  area]? 

Are  you  familiar  with  the  UN? 

Is  it  possible  to  get  your  frequencies? 

Do  you  have  radio  communications 
with  Capt  Smith? 

Do  you  live  in  this  area? 

And  who  is  your  commander? 

How  long  you've  been  police  now? 

Do  you  like  listening  to  music? 

What's  the  road  name? 

There's  been  fighting  then? 

Do  you  find  that  the  UN  is  neutral? 

How  long  have  you  been  in  town? 

Your  boss,  what  is  his  name? 

A  lot  of  PLA  guys  in  this  area  right? 

Where's  your  higher  commander? 

How  many  terrorists  were  killed  then? 

Do  you  have  a  phone  on  you? 

Who  is  the  mayor  of  your  city? 

Where  you  live,  this  area? 

What  is  your  boss’s  rank? 

And  how  far  is  the  town? 

Do  you  know  about  other  people  in 
the  area  being  in]ured  by  police? 

How  do  you  move  around?  By 
vehicle? 

Where  was  the  [civilian's]  farm? 

You  have  a  hospital  over  there  [at  the 
station]? 

And  your  village  is  where? 

When  we  get  to  the  station,  they'll  be 
cleaned  up  and  kept  in  a  nice  place? 

Where  are  they  standing  trial? 

Which  prison  is  it? 

And  where  do  you  work  Sgt? 

Table  2  shows  the  closed-ended  questions  seeking  situation  specific  information  that  trainees  asked 
throughout  the  three  field  studies. 


Table  2:  Closed-ended  questions  seeking  situation  specific  information 


MEFSI 

MEFS  II 

MEFS  III 

These  people  are  involved  in  this 
problem?  [re:  civilians  involved  in 
terrorism] 

Now  this  terrorist  organization  that 
killed  your  brother,  they  are  from 
around  this  area,  1  understand? 

Is  your  government  aware  you  are 
holding  these  people  and  are  you 
sanctioned  to  do  that? 

How  often  do  you  bring  people  out 
here  and  how  many? 

When  you  picked  them  up  did  you  tell 
their  families  where  you  were  taking 
them? 

How  long  have  you  guys  been  doing 
an  interrogation  like  this? 

So  is  this  your  normal  way  of 
interrogating? 

You  got,  you  got  a  radio?  Can  you 
contact  [your  Captain]? 

Are  you  having  problems  with  these 
people? 

Are  there  other  investigations 
happening? 

Are  you  being  told  by  your  boss  to  do 
this? 

Do  you  normally  treat  [the  civilians] 
this  way? 

And  [your  commanding  officer  has] 
authorized  this  [investigation]? 

And  how  long  do  you  expect  these 
men  to  be  digging  here? 

Who  else  do  you  have  right  here  with 
you? 

And  then  you  take  the  prisoners  back 
to  [town]? 

Have  you  got  the  paperwork  on  you 
[proving  they're  terrorists]? 

Do  you  know  the  names  of  those 
people? 

You're  walking  back  and  forth  it  must 
be  safe. 

Is  it  safe  for  us  to  walk  [around  here], 
without  worry  about  mines? 

Is  that  all  you’re  going  to  do  is  make 
them  dig  a  hole? 

Is  there  anybody  else  around  here? 

How  many  terrorists  are  normally 
buried  here? 

How  long  are  you  going  to  keep  them 
in  custody? 

[The  civilians  are]  from  what  group? 

Are  they  prisoners  or  are  they 
working? 

Is  your  radio  down? 

Is  this  a  cemetery? 

Where  are  you  taking  them,  to  your 
station? 

[The  civilians]  are  not  soldiers? 

When  will  you  be  done,  today? 

Did  you  find  them? 

Is  everything  okay  over  there? 

You  have  terrorists  here? 

So  what's  your  name? 

No  rank?  [referring  to  civilians] 

Are  they  prisoners  of  war? 

Do  you  know  the  names  yet? 

Is  that  your  supervisor  right  there? 

[The  civilians  are]  not  military? 

Do  you  know  who  we  represent? 

You  are  part  of  the  militia? 

When  did  you  come? 

So  these  aren't  farmers? 

Are  you  in  charge  of  this  area  here? 

Do  you  have  [the  civilian's]  names? 

Have  they  had  their  trial  yet? 

You  are  from  the  WIA? 

No  communications  whatsoever? 

What's  his  name? 

Are  there  mines  around  here? 

Are  they  armed? 

Is  this  a  police  operation? 

These  people  are  your  detainees? 

When  did  you  catch  them? 

So  they're  digging  graves? 

We're  ]ust  a  military  observer  team 
meeting  a  patrol  here,  are  you  the 
patrol? 

Do  you  normally  do  this  with  people? 

Oh,  there  are  land  mines? 

Have  they  been  to  ]ail? 
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MEFSI 

MEFS  II 

MEFS  III 

And  Corporal,  he's  Eastland? 

Do  they  have  weapons? 

Who's  directing  those  guys,  you? 

Are  they  Eastlanders? 

So  this  Is  a  graveyard  here? 

Are  these  citizens  of  your  country  or 
are  they  foreigners? 

Does  your  partner  have  any  ID? 

When  are  they  standing  trial? 

What's  your  Constable's  name? 

What's  your  rank? 

So  you  view  them  as  traitors? 

Are  there  mines  In  the  area? 

Are  you  In  charge? 

Table  3  shows  the  open-ended  questions  seeking  general  information  that  trainees  asked 
throughout  the  three  field  studies. 


Table  3:  Open-ended  questions  seeking  general  information 


MEFSI 

MEFS  II 

MEFS  III 

What  is,  where's  your  general 
frequency  for  the  police  bulletin? 

Just  now  before  we  leave  just  for  our 
own  safety,  any  main  areas  of  terrorist 
activity  that  we  should  watch  out  for? 

So  in  your  opinion,  how  are  the 
prisoners  being  treated  at  the  camp, 
are  they  being  well  fed? 

So  how  did  you  do  through  the  war? 

And  if  1  wanted  to  read  up  on  this 
report,  where  would  1  go?  Is  there  a 
number  to  this? 

Tell  me  your  perspective  on  that, 
[referring  to  UN  helping  terrorists] 

How,  how  are  you  going  back  to  the 
police  station? 

Just  out  of  curiosity,  explain  how  your 
judicial  system  works. 

What  kind  of  problems  do  you  think 
the  UN  should  take  care  of? 

Go  where?  [re:  Sgt  saying  he  has  to 

go] 

What  do  you  mean?  [about  the  UN 
delivering  food  to  Eastland] 

Anything  more  you  can  tell  us,  how 
can  we  contact  you? 

And  who  would  the  police  be? 

So  you  would  like  more  [supplies], 
what  is  it  that  you  would  like? 

What  have  you  been  told  about  the 

UN? 

1  was  wondering,  do  you  know  where 
[the  inspector]  might  be  right  now? 

But  there  must  be  a  prisoner  camp 
near  by? 

Why  [can't  you  talk  to  someone  when 
their  in  custody]? 

Well  is  there  anything  else  to  talk 
about? 

Oh,  how  come  [you  can't  explain  the 
judicial  system]? 

How  are  things  in  this  area? 

How  were  you  able  to  apprehend  your 
suspects? 

Where  can  1  meet  with  you? 

Where  is  the  police  station  from  here? 

So  how  do  you  talk  to  your  inspector? 

Who  trialed  them? 

What  sort  of  terrorism? 

And  where's  the  federal  prison? 

Finally,  Table  4  shows  the  open-ended  questions  seeking  situation  specific  information  that 
trainees  asked  throughout  the  three  field  studies. 
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Table  4:  Open-ended  questions  seeking  situation  specific  information 


MEFSI 

MEFS  II 

MEFS  III 

What  kind  of  danger  do  you  mean  by 
that?  [re:  Sgt  saying  UNMOs  in 
possible  danger  by  being  there] 

Have  you  heard,  are  there  any  other 
villagers  that  are  in  this  predicament 
here? 

So  what  are  the  intentions  with  [them], 
what  are  you  going  to  do  with  them 
after  you’re  done  interrogating  them? 

It's  difficult  for  me  to  understand,  so 
maybe  you  can  explain  to  me?  [re: 

Sgt  saying  it's  valuable  work  to  punish 
the  prisoners] 

[The  police  investigation  is]  regarding 
what? 

[re:  Sgt  dealing  with  terrorists]  How  do 
you  plan  on  doing  that? 

You  want  to  give  us  a  reason  why 
there  [are]  headstones  there,  or  a 
cross? 

What's  the  writing  on  the  board  over 
there? 

What  exactly  are  you  getting  the  guys 
to  do  then? 

Who  do  you  think  the  insurgents  are? 
[re:  Sgt’s  statement  about  road  side 
bombs] 

The  hard  path  is  over  here,  why  are 
you  taking  this  dangerous  route? 

So  what’s  the  usual  process,  they 
stay  here  dig  holes  for  a  while? 

Why  are  they  yelling  that  they're  going 
to  be  killed? 

Well  how  do  you  take  orders  from  the 
chain  of  command  then? 

How  are  the  civilians  getting  out  of 
hand? 

Can  you  tell  me  maybe  what's 
happening  here? 

Sgt  what's  the  ear  piece  you  have  in 
your  ear? 

How  are  we  making  their  ]ob  hard 

Sgt?  [referring  to  the  Constables’  ]ob 
of  keeping  the  civilians  busy  working] 

Can  you  tell  me  a  bit  about  the 
investigation? 

Why  [can't  we  make  contact  with 
them]? 

What  other  organizations  would  come 
in  here  with  weapons? 

What  exactly  are  they  being  charged 
with? 

Why  is  there  a  cross  there  on  the 
ground? 

Where  are  you  going  [referring  to  the 
Sgt  going  down  to  speak  with  his 
Constable]? 

Breaking  [the  civilians]  down  for 
what? 

And  why  are  they  doing  [forced 
labour]? 

So  what  are  you  doing  for  these  guys 
[civilians]  you’re  making  work? 

So  what  kind  of  prisoners  do  you  have 
here  now? 

What  type  of  investigation  are  you 
carrying  out  now? 

Who  are  these  civilians? 

What's  your  plan  here  with  these 
terrorists? 

What  kind  of  police  business? 

What  are  your  intentions? 

Where  are  [the  other  prisoners]  now? 

What's  the  work  they're  doing? 

What  is  your]ob? 

Where  did  you  get  [the  civilians]? 

And  what's  their  punishment? 

What  have  we  done  to  upset  them 
[referring  to  the  civilians]? 

Why  are  they  digging  holes? 

Oh,  who  are  they  digging  graves  for? 

So  what  are  they  building  back  there? 

What  are  the  signs  all  about  here? 

Oh,  how's  the  investigation  going? 

What  are  those  two  crosses  there? 

And  why  can't  you  tell  us  the  names? 

Who  are  those  two  people? 

1  ]ust  want  to  know  what’s  going  on. 

Why  are  they  not  cooperating? 

Why  are  they  not  going  to  the  station? 

Why  do  you  have  weapons? 

Where  are  you  taking  [the  prisoners]? 

Why  can't  you  talk  to  me? 

What  are  they  digging? 

What  are  you  trying  to  find  out? 

Interesting,  what  were  those  orders? 

Okay,  so  what's  your  plan? 

These  people,  where  are  they  from? 

MEFSI 

MEFS  II 

MEFS  III 

What  kind  of  work  are  they  doing? 

And  what  are  they  burying? 

What  [are]  the  normai  procedures? 

Where  wiii  you  be  taking  them? 

How  do  you  interrogate  them? 

Where  are  the  bodies  that  they're 
burying? 

Why  not?  [re:  why  they  can't  go  iook 
at  the  hoies  the  civiiians  are  digging] 

What's  dangerous  about  [the  area]? 

What  is  the  situation  now? 

Oh  where  [are]  the  mines? 

To  provide  greater  context,  the  research  team  also  identified  a  number  of  themes  and  sub-themes 
with  respect  to  the  questions  that  UNMO  trainees  asked  during  the  human  rights  violation  scenario. 
Table  5  categorizes  the  themes  and  sub-themes,  and  includes  example  questions  related  to  it. 
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Table  5:  Common  questioning  themes  and  prototypic  examples 


Themes 

Sub  Themes 

Examples 

Identification 

Names  (People, 
Places) 

Who  Is  your  commander?  What's  the  name  of  the  town?  What's  the  road  name? 
What's  your  name?  What's  your  Constable's  name?  Who  do  you  think  the 

Insurgents  are  [of  the  road  side  bombs]?  Who  are  those  two  people? 

Ranks 

What  Is  your  boss’s  rank?  What's  your  rank?  No  rank  [referring  to  civilians]? 

Identity 

And  who  would  the  police  be?  You  wouldn't  have  some  ID  on  you  by  any  chance? 

Are  these  citizens  of  your  country  or  are  they  foreigners?  And  Corporal,  he's 

Eastland?  Does  your  partner  have  any  ID?  Right  and  do  you  know  who  we 
represent?  [The  civilians  are]  from  what  group? 

Location-based 

Places  (Buildings, 
Towns) 

Where  do  you  work  Sgt?  Where  Is  the  police  station  from  here?  But  there  must  be  a 
prisoner  camp  near  by?  How  far  Is  the  town?  Where's  the  federal  prison? 

People 

Where  are  the  bodies  that  they're  burying?  Where's  your  higher  commander? 

Residence 

Do  you  live  In  this  area?  Where  are  these  people  from? 

Rationale-based 

Civilian  Activity 

It's  difficult  for  me  to  understand,  so  maybe  you  can  explain  to  me  [how  It’s  valuable 
work]?  What's  the  work  they're  doing?  What's  their  punishment?  And  why  are  they 
doing  [forced  labour]?  What  are  they  digging?  What  Is  the  hole  being  dug  for?  And 
how  long  do  you  expect  these  men  to  be  digging  here? 

Gravesite 

You  want  to  give  us  a  reason  why  there  [are]  headstones  there,  or  a  cross?  What's 
that  marker?  That's  a  graveyard  Isn't  It? 

General 

Can  you  tell  me  what's  going  on  here?  What’s  happening  here? 

Time-based 

Before 

When  you  picked  them  up  did  you  tell  their  families  where  you  were  taking  them? 
When  did  you  come?  When  did  you  catch  them?  Have  they  been  to  ]all? 

After 

And  the  ]all  where  they're  being  housed  Is  at  the  police  station?  When  are  they 
standing  trial?  And  what's  going  to  happen  to  these  guys  you  call  murderers?  So 
what  are  the  Intentions  with  [them],  what  are  you  going  to  do  with  them  after  you’re 
done  Interrogating  them?  Where  are  you  taking  [the  prisoners]? 

Investigation 

Investigation 

How  were  you  able  to  apprehend  your  suspects?  Can  you  tell  me  a  bit  about  the 
Investigation?  What  exactly  are  they  being  charged  with?  [The  police  Investigation 

Is]  regarding  what?  Are  there  other  Investigations  happening? 

Interrogation 

What  sort  of  Interrogation  are  you  proceeding  with?  How  do  you  Interrogate  them? 
How  long  have  you  guys  been  doing  an  Interrogation  like  this? 

Security 

UNMOs 

Just  now  before  we  leave  ]ust  for  our  own  safety,  any  main  areas  of  terrorist  activity 
that  we  should  watch  out  for?  Is  It  safe  for  us  to  go  straight  Into  the  village?  Are 
there  mines  In  the  area?  What  kind  of  danger  do  you  mean  by  that?  You're  walking 
back  and  forth.  It  must  be  safe? 

Sgt 

The  hard  path  Is  over  here,  why  are  you  taking  this  dangerous  route?  Are  they 
armed? 

Civilians 

You'll  take  him  back  to  the  hospital? 

Personal/Rapport 

building 

Sgt 

So  how  did  you  do  through  the  war?  How  long  have  you  been  working  for  the 
police?  Do  you  have  children  at  all?  Family  In  the  area?  Do  you  like  listening  to 

music?  You  married?  How  long  have  you  not  had  food  now? 


3.3  Mil  Obs  Training  Evaluation  Framework 

The  research  team  received  documentation  from  the  Peace  Support  Training  Centre  that  detailed 
the  most  critical  performance  objectives  in  the  Military  Observer  (MilObs)  Course.  This 
documentation  detailed  the  kinds  of  skills  that  military  personnel  were  meant  to  acquire  while  on 
the  course  and  the  means  in  which  PSTC  Instructors  impart  those  skills.  The  following  tables 
identify  these  most  critical  performance  objectives  (i.e.,  outcomes)  in  the  PSTC  MilObs  course  and 
the  activities  meant  to  ensure  these.  The  first  core  performance  objective,  employ  personal 
sustainment  skills,  and  the  knowledge/skills  and  Mil  Obs  course  activities  associated  with  it  are 
detailed  in  Table  6. 


Table  6:  Mil  Obs  course  activities  in  support  of  employing  personal  sustainment 

skills 


Performance 

Objective 

Standard 

Knowledge/Skills 

Courses 

Employ  Personal 
Sustainment  Skills 

Personal  Security 

•  Knowledge  for  operating  In  a  mined 
environment 

•  Knowledge  of  preventative  measures 
to  operate  In  a  hostage/hijack 
situation 

•  Knowledge  of  survival  skills  for 
hostage/hijack  situation 

•  Ability  to  make  risk  and  threat 
assessment 

•  Knowledge  of  evacuation  and 
personnel  recovery  procedures 

•  Field  craft 

•  Survival  lAW 

•  Security  -  risk  &  threat 

•  Security  -  protection 

•  Law  of  armed  conflict 

•  Use  of  force 

•  Explosive  threat 
awareness 

•  Mission  terrain 

•  Mission  Intelligence 

•  Mission  operations 

First  Aid 

•  Knowledge  of  MIlOb  medical  kit 

•  Knowledge  of  triage  and  vital  signs 

•  Ability  to  treat  ballistic  and  blast 

Injuries 

•  Combat  related  first 
aid 

Health  &  Hygiene 

•  Ability  to  prepare  for  austere 
environments 

•  Knowledge  of  preventative  medicine 

•  Ability  to  react  to  health  and  hygiene 
symptoms 

•  Preventative  medicine 

Administration  & 
Support 

•  Knowledge  of  CF  support 
mechanisms  (e.g.,  allowances  and 
benefits) 

•  Knowledge  of  UN  administration 

•  Knowledge  of  Peace  Support 

Operation 

•  Knowledge  of  ROES 

•  Cultural  Awareness 

•  General  knowledge  of 
UN 

•  Human  rights 
awareness 

•  Cultural  Awareness 
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The  desired  course  outcomes  listed  for  employing  personal  sustainment  skills  can  be  directly 
trained  and  assessed  by  the  PSTC  Instructors.  Another  core  performance  objective,  employ  military 
observer  team  skills,  and  the  knowledge/skills  and  Mil  Obs  course  activities  associated  with  it  are 
detailed  in  Table  7. 


Table  7:  Mil  Obs  course  activities  in  support  of  employing  military  observer  team 

skills 


Performance 

Objective 

Standard 

Knowledge/Skills 

Courses 

Employ  Military 
Observer  Team 

Skills 

Act  as  a  MilObs 
team  member 

•  Ability  to  manage  critical  incidents 

•  Ability  to  manage  combat  stress 

•  Ability  to  engage  in  conflict  resolution 

•  Knowledge  of  briefing  techniques 

•  Knowledge  of  roles  and 
responsibilities 

•  Personal  conduct 

•  PSO  environment 

•  PSO  environment  - 
practical 

•  Stress  management 

•  Duties  of  a  Military 
Observer  (com  ms 
equipment;  UN 
comms;  Ex  radio 
check;  Staff  duties; 

FFE) 

Manage  a  MilObs 
team 

•  Ability  to  organize  team  site 

•  Ability  to  operate  communications 
equipment 

•  Managing  diversity 

Direct  a  MilObs 
team 

•  Situational  awareness 

•  Ability  to  manage  mission  information 

•  Ability  to  balance  team  resources 

•  Ability  to  manage  team  dynamics 

•  Duties  of  a  Mil  Ob 
(Handling  information) 

The  desired  course  outcomes  listed  for  employing  military  team  skills  can  be  directly  trained  and 
assessed  by  the  PSTC  Instructors.  Another  core  performance  objective,  conduct  military  observer 
functions,  and  the  knowledge/skills  and  Mil  Obs  course  activities  associated  with  it  are  detailed  in 
table  8. 


Table  8:  Mil  Obs  course  activities  in  support  of  conducting  military  observer 

functions 


Performance 

Objective 

Standard 

Knowledge/Skills 

Courses 

Conduct  Military 

Observer 

Functions 

Conduct  patrols 

•  Ability  to  navigate  of  land,  air, 
and  sea 

•  Ability  to  use  4x4  vehicle 

o  Maintenance 
o  Security  when  driving 
o  Driving  techniques 

•  Patrolling  theory 

•  Patrolling  -  practical 

•  Navigation 

•  Navigation  (dismounted) 

•  Tactical  driving 

Conduct  observation 

•  Ability  to  identify  military  AFV/AC 

•  Ability  to  monitor  use  of  force 

•  Ability  to  react  to  human  rights 
violations 

•  Ability  to  conduct  static  and 
mobile  observations 

•  Ability  to  conduct  mounted  and 
dismounted  observations 

•  Observation  theory 

•  Observation  -  practical 

Investigations 

•  Ability  to  conduct  crater  analysis 

•  Investigation 

•  Investigation  -  crater 
analysis 

Inspections/verifications 

•  Ability  to  handle  small  arms 

•  Inspection/verification 

•  Foreign  weapons 

Perform  disarmament, 
demobilization  and 
reintegration  activities 

•  Ability  to  perform  DDR  activities 

•  DDR 

Again,  the  desired  course  outcomes  listed  for  conducting  military  observer  functions  can  be 
directly  trained  and  assessed  by  the  PSTC  Instructors.  Another  core  performance  objective, 
employing  communication  skills,  and  the  knowledge/skills  and  Mil  Obs  course  activities  associated 
with  it  are  detailed  in  table  9. 


Table  9:  Mil  Obs  course  activities  in  support  of  employing  communication  skills 


Performance 

Objective 

Standard 

Knowledge/Skills 

Courses 

Employ 

Communication 

Skills 

Ability  to  conduct  media 
interviews 

•  Media  awareness 

•  Multicultural 
communication  skills 

Ability  to  conduct 
negotiations 

•  Counter  lED 

Ability  to  perform  liaison 
tasks 

•  Liaison 
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Like  the  other  performance  objectives,  employing  communication  skills  can  be  directly  trained  and 
assessed  by  the  PSTC  Instructors. 

PSTC  Instructors  mentioned  a  number  of  other  vital  skills  for  Mil  Obs  that  they  believed  could  be 
coached,  but  could  not  be  directly  trained  or  assessed.  Table  10  highlights  the  coachable  skills  for 
individuals  and  for  teams. 


Table  10:  Required  coachable  skills  for  Mil  Obs  trainees 


COACHABLE  SKILLS 

As  an  individual 

• 

Adaptability 

• 

Flexibility 

• 

Patience 

• 

High  tolerance  for  frustration 

• 

Open-mindedness 

• 

Self-motivator 

• 

Initiative 

• 

Self-confidence 

• 

Perseverance 

• 

Self-reliance 

• 

Sense  of  humour 

In  a  team 

• 

Communication 

• 

Tact 

• 

Tolerance 

• 

Cohesiveness 

• 

Diplomacy 

• 

Understanding 

• 

Cultural  sensitivity 

• 

Cooperation 

• 

Global  awareness 

• 

Situational  awareness 

• 

Unbiased/non-judgmental 

PSTC  Instructors  believed  that  these  coachable  skills  were  critical  for  Mil  Obs  to  have  as  an 
antecedent  to  learning  and  developing  the  core  performance  objectives  (i.e.,  personal  sustainment. 
Mil  Obs  team  skills.  Mil  Obs  functions,  and  communication  skills).  One  particular  attribute  that 
PSTC  Instructors  thought  was  vital  was  adaptability.  Indeed,  many  of  the  course  exercises  are 
constructed  in  order  to  test  this  attribute.  It  will  be  important  for  the  evaluation  to  be  able  to 
determine  which  coachable  skills  influence  the  development  of  the  core  performance  objectives. 


4.  Way  Ahead 


The  next  step  for  the  PSTC  training  evaluation  is  for  the  research  team  to  establish  measures  based 
on  a  curriculum  and  training  analysis  of  the  PSTC  Mil  Obs  course.  The  information  gathered  in  this 
analysis  will  provide  the  necessary  information  to  generate  quantitative  and  qualitative  instruments 
to  effectively  evaluate  the  PSTC  training  program  in  regards  to  its  relevance  to  actual  UN 
operations.  Data  collection  is  meant  to  assess  just  how  relevant  those  outcomes  and  training 
activities  are  for  UNMOs  in  operations.  The  evaluation  will  determine  how  well  the  training  that 
UNMOs  receive  at  the  PSTC  during  the  Mil  Obs  course  translates  into  actual  operational 
experience. 

To  complete  the  evaluation,  the  research  team  proposes  data  collection  at  three  points.  Time  one 
would  include  reviewing  trainees’  feedback  regarding  the  course  that  is  currently  provided  to  PSTC 
at  the  end  of  each  activity  and  the  course.  As  well,  the  research  team  will  administer  the  evaluation 
survey.  Time  two  data  collection  would  be  conducted  part  way  through  the  trainees’  deployment. 
Finally,  time  three  would  be  when  trainees  return  from  their  tours.  Data  collection  would  include 
not  only  the  evaluation  survey,  but  also  an  interview  in  order  to  collect  qualitative  data.  To  increase 
accessibility  and  minimize  costs  to  the  project,  administering  an  on-line  survey  is  suggested. 
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Field  Study  1 
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Field  Study  1 


Purpose 


Explored  impact  of  moral 
intensity  on  the  moral  and  ethical 
decision-making  (MEDM)  and 
negotiation  behaviour  in  a  human 
rights  violation  training  scenario 
Moral  intensity  (Jones,  1991)  is 
the  perceived  salience  or  vividness 
of  a  moral  issue  (relevant  factors 
include  proximity,  magnitude  of 
consequences,  immediacy,  etc.) 

To  manipulate  trainees’  levels  of 
moral  intensity,  this  study  varied 
the  physical  proximity  of  the 
female  victim 
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fField  Study 
*  Conditions 


1  -  Experimental 


High  Moral 

Intensity  Condition 

Baseline  Moral 
Intensity  Condition 

At  5  minutes  into  the 
scenario,  the  female  civilian 
rushed  toward  the  trainees 
(stopping  within  4  feet  and 
making  eye  contact) 
appealing  to  them  for  their 
help. 

At  5  minutes  into  the 
scenario,  the  female  civilian 
appeals  to  trainees  for  their 
help  the  same  way  as  in  the 
high  moral  intensity  condition 
but  stays  at  the  gravesite 
(approximately  60  feet  away). 

_ 
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of  High  MI  on  Behaviour 

1.  More  likely  to  remain  engaged  in  the  scenario  (i.e., 
no  team  in  the  high  moral  intensity  condition  left 
the  site,  despite  the  Sgt’s  demands  they  do  so). 

2.  More  frequent  proactive  negotiation  behaviours 
(e.g.,  maldng  more  attempts  to  make  contact  with 
the  civilians  and  to  diffuse  the  situation  as  well  as 
taking  notes  and  asking  for  names). 

3.  More  frequent  negative  relationship-building 
behaviours,  (e.g.,  acting  without  permission, 
provoking  the  Sgt,  being  opinionated  and 
judgemental,  threatening  the  Sgt  with  future 
consequences,  being  sarcastic  or  condescending.) 
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jother  Findings 


I 


1.  Teams  that  watched  the  police  lead  the  civilians 
into  the  forest  were  less  satisfied  with  the  quality  of 
their  response  and  with  the  outcome  than  teams 
that  either  followed  or  left  the  situation  (i.e., 
disengaged).  (Failure  to  take  a  definitive  stand?) 


2.  Team  leaders  rated  the  police  behaviour  as  more 
immoral  than  did  team  members,  suggesting  the 
greater  involvement  of  the  leaders  in  the 
negotiation  might  have  influenced  their  perceptions 
of  the  police’s  behaviour. 
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jother  Findings 


II 


3.  Team  leaders  also  ascribed  significantly  more 
responsibility  to  the  police  for  the  events  unfolding 
during  the  scenario  than  did  other  team  members. 

4.  Team  leaders  were  less  likely  to  want  to  change 
their  behaviour  if  they  could  redo  the  scenario  than 
team  members. 
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Field  Study  2 
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•  Varied  the  anger  and 
agitation  of  trainees’ 
negotiation  partner  (Sgt) 
to  explore  impacts  on  their 
negotiation  behaviour  and 
perceptions 


•  Could  “take  on”  the  anger 
of  the  Sgt  themselves  or 
might  strategically  adjust 
their  own  negotiation 
behaviour  to  “bring  him 
down” 
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frield  Study  2  -  Experimental 
*  Conditions 


Angry  Condition 

Neutral  Condition 

The  police  Sgt  exhibited 
aggressive,  uncooperative 
behaviours  associated  with 
anger  (e.g.,  shouting, 
interrupting,  etc.). 

The  police  Sgt  acted  more 
neutrally  (i.e.,  less  angry) 
and  was  more  yielding. 
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of  Anger  on  Behaviour 

1.  Trainees  worked  harder  to  diffuse  the  situation,  they 
were  more  responsive  to  Sgt’s  concerns,  they  showed 
more  empathy  for  the  victims,  they  asked  permission 
more,  and  they  complied  more  with  the  Sgt  when  he 
made  demands  of  them. 

2.  However,  they  also  showed  more  negative 
relationship  building  behaviors,  such  as  being 
condescending/insulting,  expressing  critical  opinions 
about  police,  threatening  the  Sgt  with  future 
consequences,  circumventing  his  authority,  and 
opposing  his  demands.  To  some  extent,  trainees 
mirrored  the  Sgt’s  emotion  (social  contagion  effect). 
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jother  Findings 


I 


1.  Teams  that  watched  the  police  lead  the  civilians 
into  the  forest  rated  their  anger  toward  the  Sgt 
marginally  higher  than  those  who  either  left  or 
followed  the  police  and  civilians. 


2.  Team  leaders  were  less  likely  to  want  to  change 
their  behaviour  if  they  could  redo  the  scenario  than 
team  members. 
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How  does  “working  as  a 
team”  influence  the 
negotiation  performance  of 
teams  (e.g.,  does  team 
“brainstorming”  improve  the 
quality  of  the  negotiation?) 

How  does  having  an 
opportunity  to  contribute 
more  influence  one’s  view  of 
the  negotiation  (e.g., 
satisfaction  with  outcome)? 


Field  Study  3  - 


Purpose 


ITield  Study 
'  Conditions 


3  -  Experimental 


“Team”  Condition 

Baseline  Condition 

After  3 
minutes 
into  the 
scenario 

“I  thouqht  vou  quvs  were 
supposed  to  work  as  a  team.  I 

gotta  go  talk  to  my  partner.  I 
don’t  care  if  you  talk,  but  don’t 
move  around.” 

“I  gotta  go  talk  to  my  partner.  I 
don’t  care  if  you  talk,  but  don’t 
move  around.” 

After  10 
minutes 
into  the 
scenario 

“We  gotta  leave  very  soon  to  get 
these  people  to  the  police 
station.  I  gotta  talk  to  my 
constable.  You  talk  to  vour 
team.” 

“We  gotta  leave  very  soon  to  get 
these  people  to  the  police  station.  I 
gotta  talk  to  my  constable.” 

HUMANS)'Sr£.WS 
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of ‘‘Team”  on  Behaviour 

1.  Teams  shared  more  information  amongst  themselves. 

2  Team  members  contributed  at  least  as  much  as  team 
leaders  (and  some  contributed  more). 
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Impact  of  “Team” 


on  Behaviour 


3.  Team  behaviours  observed  in  team  huddles  were: 

•  information  exchange 

•  role  assigning  and  confirming 

•  back-up  (especially  when  team  leader  was  firom  a  foreign 
country). 


4.  8  of  14  teams  worked  as  a  team  to  make  final  decision 
(re:  the  outcome  in  the  2*“*  team  huddle).  The  other  6 
teams  either  did  not  say  anything  in  the  2"**  huddle 
(n=2)  or  gave  suggestions  re:  negotiation  (n=4). 
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1.  Relative  to  leaders,  team  members  thought: 

•  they  had  significantly  less  personal  input  into  tlie  negotiation 

•  their  ideas  were  used  significantly  less 

•  their  ideas  were  significantly  less  valued 

•  they  were  more  likely  to  wish  to  redo  scenario. 


2.  Teams  who  demonstrated  back-up  behaviour  strongly 
agreed  on  questionnaire  measures  that  their 
teammates  provided  assistance  when  necessary;  thus, 
they  were  attuned  to  this  particular  activity. 
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Lessons  Learned 


*  Face-to-face  interaction  with  a  victim  appears  to 
increase  trainees’  moral  commitment  to  remain  with 
the  victim. 

*  It  is  possible  to  “take  on”  the  emotional  state  of  one’s 
negotiation  partner,  shaping  negotiation  behaviours. 

*  Trainees  often  exhibited  negative  relationship  building 
behaviours,  (e.g.,  expressing  a  critical  opinion  of  the 
police’s  behaviour),  when  negotiating  with  the  Sgt. 
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Lessons  Learned  cont 


•  Team  members  and  team  leaders  may  have  very 
difTerent  experiences,  depending  on  the  level  of  input 
each  has  during  the  negotiation. 

•  Multicultural  teams  may  show  greater  team  back-up 
behaviours. 

•  Having  an  opportunity  to  re-group  as  a  team  may 
foster  improved  decision  making  and  negotiation  as 
information  becomes  shared  across  the  team. 
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PSTC  during  the  Mil  Obs  course  translates  into  actual  operational  experience.  This  report 
concludes  with  possible  next  steps  toward  the  evaluation  and  a  plan  for  data  collection. 

(U)  Un  Programme  de  recherches  appliquees  (PRA)  en  cours  de  la  Section  de  la  recherche 
concertee  sur  le  rendement  et  I'apprentissage  (Section  RCRA)  a  Recherche  et 
developpement  pour  la  defense  Canada  (RDDC)  Toronto  explore  la  prise  de  decisions 
morales  et  ethiques  (PDME)  dans  le  cadre  des  operations  des  Forces  canadiennes  (FC) 
(16kk,  16re02).  Les  etudes  sur  le  terrain  menees  au  Centre  de  formation  pour  le  soutien 
de  la  paix  (CFSP)  a  la  base  des  Forces  canadiennes  (BFC)  de  Kingston  sont  I’un  des 
importants  champs  d’interet  de  ce  programme  de  recherches.  Les  membres  des  FC  qui  se 
preparent  a  partir  en  mission  en  tant  qu’observateurs  militaires  des  Nations  Unies  (UNMO 
ou  Ob  mil)  entreprennent  un  certain  nombre  d’activites  d’entramement  destinees  a 
ameliorer  et  developper  les  competences  propres  aux  Operations  de  paix  des  Nations 
Unies  (OPNU).  Pendant  un  mois,  les  membres  des  FC  suivent  un  programme  de 
preparation  intensif  faisant  appel  a  une  combinaison  de  formation  en  classe  et 
d’instruction  en  situation.  L’un  des  exercices  proposes  met  en  scene  une  situation  de 
violation  des  droits  de  la  personne.  Dans  ce  scenario,  les  stagiaires  non  armes  tentent  de 
negocier  les  vies  de  civile  qui  sont  victimes  de  violences  physiques  et  verbales  de  la  part 
de  deux  policiers  armes.  Notre  programme  de  recherche  a  examine  jusqu’ici  un  certain 
nombre  de  processus  psychologiques  au  cours  de  ce  scenario,  tels  que  les  aspects  de 


I’intensite  morale  (proximite  physique  de  la  victime,  MEFS  I),  emotion  (MEFS  II)  et  travail 
d’equipe  (MEFS  III).  Les  travaux  fournis  actuellement  par  les  instructeurs  du  CFSP 
demandant  des  documents  qui  sent  tires  des  resultats  de  ces  etudes. 

Premierement,  nous  avons  cree  une  serie  de  diapositives  PowerPoint  qui  montrent  en 
detail  les  decouvertes  les  plus  importantes  des  trois  etudes  sur  le  terrain.  Deuxiemement, 
nous  avons  redige  la  liste  des  questions  ouvertes  (a  reponse  libre)  et  fermees  (a  reponse 
preetablie)  que  les  stagiaires  ont  posees  durant  le  scenario  lors  des  trois  etudes  (MEFS  I, 
MEFS  I  et  MEFS  III).  Cette  liste  est  fournie  afin  de  mettre  en  evidence  I’instruction  du 
CFSP  aux  stagiaires,  c’est-a-dire  mettre  en  valeur  rutilisation  de  questions  ouvertes  pour 
provoquer  une  connaissance  de  la  situation.  Des  tableaux  contenant  des  exemples  precis 
de  questions  ouvertes  et  fermees  destinees  a  obtenir  des  renseignements  generaux  et 
particuliers  ont  ete  elabores.  On  a  egalement  prepare  un  tableau  distinct  dans  lequel  les 
themes  communs  ont  ete  extraits  de  ces  questions;  ce  tableau  contient  aussi  des 
exemples  prototypiques  des  questions  posees  par  les  stagiaires.  La  troisieme  et  derniere 
section  du  present  rapport  comprend  les  activites  preparatoires  qui  jettent  les  bases  d’une 
evaluation  de  I’instruction  du  CFSP.  Plus  precisement,  cette  evaluation  a  pour  but  de 
decouvrir  dans  quelle  mesure  I’instruction  regue  par  les  UNMO  au  CFSP  dans  le  cadre  du 
cours  d’observateur  militaire  se  traduit  en  une  reelle  experience  operationnelle.  Le  present 
rapport  se  termine  en  presentant  les  prochaines  etapes  menant  a  cette  evaluation  ainsi 
qu’un  plan  pour  la  collecte  de  donnees. 
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